1. Field of the Invention
This invention relates to motion compensated video signal processing.
2. Description of the Prior Art
Motion compensated video signal processing is used in video processing applications such as television standards conversion or video to film conversion. An example of a previously proposed motion compensated video processing apparatus is described in the British Published Patent Application number GB-A-2 231 749, in which pairs of temporally adjacent images (fields or frames) of an input digital video signal are processed to generate corresponding sets of motion vectors. The processing is carried out on discrete blocks of the images, so that each motion vector represents the inter-image motion of the contents of a respective block. After further processing, the set of motion vectors is supplied to a motion compensated interpolator which interpolates an output image from the pair of input images, taking into account the motion between the input images.
In the above apparatus the motion vectors are generated as follows. The first stage is that individual search blocks within the input image temporally preceding a required output image are compared with a plurality of such blocks in a search area of the input image temporally following that output image. For each such comparison a correlation test is performed involving the calculation of the absolute luminance difference between pixels in corresponding positions in the respective blocks in the temporally preceding and temporally following input images. These absolute luminance differences are then summed to generate a correlation value indicative of the correlation between the two blocks being compared. The block in the search area of the temporally following image for which the correlation value is lowest is taken to be correlated with the search block in the temporally preceding image, and a motion vector is generated representing image motion from the search block to that block in the search area.
A problem which can occur with previously proposed motion compensated video signal processing apparatus will now be described with reference to FIGS. 1 to 5 of the accompanying drawings.
FIG. 1 is a one-dimensional view of an object 10 moving at a velocity V between two consecutive fields (f0, f1) of an input video signal. As described above, the input field f0 is arbitrarily divided into a grid of search blocks (e.g. the blocks 20, 30 shown in FIG. 1) and each search block is compared with a respective search area 40 comprising a plurality of such blocks, in the input field f1. A motion vector 50 is generated in dependence on the position in the search area 40 having the highest correlation with (i.e. the least difference from) the search block 20.
FIG. 2 illustrates an array of motion vectors generated from the search blocks of the input field f0 of FIG. 1. In this example, a motion vector V is generated from the search block 20, to represent the motion of the object 10 between the input f0 and the input field f1. Surrounding search blocks in the input field f0 do not contain moving objects and so a zero motion vector is generated from those search blocks.
A vector reduction process is employed to assign a group of motion vectors to each of a plurality of blocks of an output field (corresponding in positions to respective search blocks of the input field f0). This group of motion vectors is selected from, in the following order of preference,
1. The zero motion vector; PA1 2. The motion vector generated from the corresponding search block of the input field f0 (the "local" motion vector); motion vectors generated from surrounding search blocks of the input field f0 ("neighbouring" motion vectors); and PA1 4. A predetermined number of most frequently occurring distinct motion vectors from the entire input field f0 ("global" motion vectors).
Each pixel of each block of the output field is interpolated from the two input fields using one of the motion vectors from the group assigned to that block of the output field.
In the previously proposed apparatus described above, groups of four motion vectors are assigned to each block of the output image by the vector reduction process. In order to simplify the following explanation, however, FIG. 3 shows a case in which groups of two motion vectors are assigned to the blocks of the output image. A block 60 (corresponding in position to the search block 20 of the input field f0) is assigned the motion vectors V and zero. The motion vector V is also assigned (as a "neighbouring" motion vector) to the immediately adjacent blocks 70, 80 of the output image. The remaining blocks of the output image are not assigned the motion vector V.
FIG. 4 illustrates the interpolation of the output image when the temporal position of the output image is three tenths of the way between the input field f0 and the input field f1. In other words, if the input field f0 is considered to occur at a time t=0 and the input field f1 at a time t=1, then the output field occurs at t=0.3.
In order to represent the motion of the object 10, the object 10 should be interpolated at a position 90 in the output field. The position 90 falls within the block 80 of the output field for which the motion vector V was assigned during vector reduction. The object 10 can therefore be correctly interpolated using the motion vector V.
In contrast, FIG. 5 illustrates the interpolation of an output field at a temporal position t=0.7. In this case, the position 100 at which the object 10 should be interpolated falls within a block 110 of the output field for which the motion vector V has not been assigned. The object 10 cannot be correctly interpolated in the output field because the motion vector V is not available for its interpolation.
Accordingly, despite the use of vector reduction which allows blocks of the outputs fields to "borrow" more suitable motion vectors from other blocks, situations (such as that illustrated in FIG. 5) can arise in which the motion vector required for correct interpolation of a portion of the output field is not made available by vector reduction. This can be a particular problem for objects moving rapidly between two consecutive input fields, in which case the motion vector representing that motion may be significantly larger than the size of the search block.